Imaging system for vehicle

ABSTRACT

A vehicular imaging system includes a camera with an imaging sensor. A red-pass spectral filter is disposed at each pixel of a first set of photosensing pixels, a blue-pass spectral filter is disposed at each pixel of a second set of photosensing pixels, a green-pass spectral filter is disposed at each pixel of a third set of photosensing pixels, and each pixel of a fourth set of photosensing pixels senses at least near infrared radiation. A near infrared illumination source is operable to emit non-visible near infrared illumination to enhance nighttime viewing by the camera. Outputs of the pixels of the first, second and third sets of photosensing pixels are processed for at least one of (i) a lane departure warning system and (ii) an object detection system. Output of the photosensing pixels of the fourth set of photosensing pixels is processed for a night vision system.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 14/562,826, filed Dec. 8, 2014, now U.S. patent Ser. No.10/003,755, which is a continuation of U.S. patent application Ser. No.13/896,845, filed May 17, 2013, now U.S. Pat. No. 8,908,040, which is acontinuation of U.S. patent application Ser. No. 12/681,204, filed May28, 2010, now U.S. Pat. No. 8,446,470, which is a 371 national phasefiling of PCT Application No. PCT/US08/78700, filed Oct. 3, 2008, whichclaims the benefit of U.S. provisional application Ser. No. 60/977,430,filed Oct. 4, 2007, which is hereby incorporated herein by reference inits entirety.

FIELD OF THE INVENTION

The present invention relates to vision systems for vehicles and, moreparticularly, to an imaging sensor or camera for a vision or imagingsystem of a vehicle.

BACKGROUND OF THE INVENTION

Traditional color imagers or image sensors or pixelated imaging arraysuse a Bayer pattern of pixels and pixel filters, which has ared-green-green-blue (R-G-G-B) pixel/filter configuration (such as shownin FIG. 1). In such a pixelated array, the sensor includes individualoptical filters that transmit red, green or blue colors and that aredisposed at or coated on the individual pixels. Thus, there is a “redpixel” 12 a, a “blue pixel” 12 b and two “green pixels” 12 c arranged toform a 2×2 sub-array 10 that is repeated over the pixelated array.

The three color filters (R, G, and B) not only pass ranges ofwavelengths or spectral bands that are corresponding to red, green andblue colors, they also pass through a significant amount of wavelengthsin the infrared (IR) or near infrared (NIR) region or band of thespectrum. Therefore, the color imager sensitivity or quantum efficiencyspectrum typically has a rich IR or NIR response even with the R, G, andB color pixels. For example, a typical silicon CMOS color sensor'sspectrum response is shown in FIG. 2. The IR response of the R, G and Bpixels is comparable or higher than the pixels' response of visiblespectrum. The IR light from the environment thus may wash-out the colorresponse in the visible spectrum and thus may distort the image colorreproduction. This is often referred to as IR contamination. In atraditional color camera, in order to reproduce a true color image, anIR cut-off filter is usually used to cut off or reduce light or energyat or in the IR band or region of the spectrum so as to allow only (orsubstantially only) the visible light to pass through the filter so asto be imaged by the RGGB pixels, in order to reduce or limit orsubstantially eliminate the IR contamination. Such an IR cut-off filteris typically made of multilayer coatings on a glass or plastic element,such as a flat glass plate, that is added on to a lens assembly of theimager or onto a surface of a lens element of the lens assembly of theimager. The coating process and added material increase the cost of thelens, sometimes significantly.

On the other hand, for IR imaging applications that have the mainspectrum of interest in the NIR region of the spectrum, where siliconCCD or CMOS can provide good quantum efficiency, one may need to cut thevisible spectrum off from the light reaching the imager. A long passfilter that is a multilayer coating on a glass or plastic flat plate istypically used to let through NIR light only. Security cameras and somespecial industrial machine vision systems are among the applications ofsuch a filter-imager configuration. For cameras that operate in both dayand night for such applications, mechanical switches of IR cutofffilters and IR pass filters are used on silicon CMOS or CCD cameras. Theadditional cost of two filters and mechanical switches, as well as thereliability of the moving mechanical switch, make such systemsundesirable, and such a design is not suitable for automotive orconsumer digital cameras or cell phone cameras.

SUMMARY OF THE INVENTION

The present invention provides an imaging sensor or camera having apixel configuration that combines red (R), green (G), and blue (B)sensing pixels with an infrared (IR) sensing pixel. The presentinvention provides de-mosaicing techniques for processing the R-G-B-IRimager pixels. The imaging sensor or camera of the present invention issuitable for use in automotive camera products, such as for vision-basedsystems, such as headlamp control systems, reverse aid systems, lanedeparture warning systems, object detection systems, rain sensorsystems, front collision warning, night vision and/or the like. Theimaging sensor or camera of the present invention may also be suitablefor use in security cameras, consumer digital cameras, cell phonecameras and industrial machine vision systems and/or the like.

Therefore, the imaging sensor of the present invention can provide bothvisible and IR imaging capability without external filters andshuttering/switching mechanism. Such a pixel configuration of R-G-B-IRpixels is applicable to various pixelated imaging array type sensors,such as CMOS, CCD and/or other image sensing devices. The imaging sensorof the present invention is suitable for applications in securitycameras, automotive cameras, consumer digital cameras, cell phonecameras, and industrial machine vision cameras, as well as other marketsand/or applications.

These and other objects, advantages, purposes and features of thepresent invention will become more apparent upon review of the followingspecification in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic of a typical R-G-G-B pattern or Bayer pattern;

FIG. 2 is a schematic of a silicon CMOS color sensor's spectrumresponse;

FIGS. 3A and 3B are schematics of R-G-B-IR pixel patterns in accordancewith the present invention;

FIG. 4 is a schematic of an array having a R-G-B-IR pixel pattern inaccordance with the present invention;

FIG. 5 is a schematic of a 2×2 window de-mosaicing method in accordancewith the present invention;

FIG. 6 is a schematic of a 2×2 rolling window de-mosaicing method inaccordance with the present invention;

FIG. 7 is a schematic of a 3×3 rolling window de-mosaicing method inaccordance with the present invention;

FIG. 8 is a schematic of a camera system combining forward viewingvision and NIR night vision display in accordance with the presentinvention;

FIG. 9 is a schematic of a rear viewing camera system with enhanced lowlight sensitivity, hardware switches normal color image and low lightNIR imaging in accordance with the present invention;

FIG. 10 is a schematic of a de-mosaicing technique for low light imagingin accordance with the present invention; and

FIG. 11 is a schematic of a rear viewing camera and display fordisplaying color images under normal lighting conditions and/ormonochrome images under low lighting conditions in accordance with thepresent invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Imaging Sensor Description

The imaging sensor of the present invention comprises a two-dimensionalpixelated imaging array having a plurality of photo-sensing pixelsarranged or disposed or established on a semiconductor substrate. Forexample, the imaging sensor may comprise acomplementary-metal-oxide-semiconductor (CMOS) or a CCD imaging sensoror device or the like, and may utilize aspects of the imaging sensorsdescribed in U.S. Pat. Nos. 5,796,094; 6,097,023; 6,320,176; 6,313,454;6,559,435; 6,831,261; 6,396,397; 5,877,897; 6,498,620; 5,670,935;6,806,452; 6,946,978; 7,123,168; 7,004,606; 7,005,974; and/or 5,550,677,and/or PCT Application No. PCT/US07/75702, filed Aug. 10, 2007 andpublished Feb. 28, 2008 as International Publication No. WO 2008/024639,and/or U.S. patent application Ser. No. 11/239,980, filed Sep. 30, 2005,now U.S. Pat. No. 7,881,496; Ser. No. 11/105,757, filed Apr. 14, 2005,now U.S. Pat. No. 7,526,103; and/or Ser. No. 10/534,632, filed May 11,2005, now U.S. Pat. No. 7,965,336; and/or PCT Application No.PCT/US03/36177, filed Nov. 14, 2003 and published Jun. 3, 2004 asInternational Publication No. WO 2004/047421; and/or PCT Application No.PCT/US06/41709, filed Oct. 27, 2006 and published May 10, 2007 asInternational Publication No. WO 07/053404, which are herebyincorporated herein by reference in their entireties.

The arrangement of pixels has a pixel pattern that is derived from theBayer pattern, but has one of the green filters or pixels replaced by aninfrared (IR) filter or pixel 12 d, such as shown in FIGS. 3A and 3B.The IR filter or pixel has an optical filter that limits or reduces orsubstantially precludes light or electromagnetic energy in the visibleband or region of the spectrum so as to pass or substantially pass IRenergy (for example, energy having a wavelength at or above about 700 nmor thereabouts) so that the pixel principally senses IR energy or NIRenergy. The red, green and blue pixels may have a similar filter asimplemented for typical Bayer pattern imaging sensors (and thusprincipally sense the respective visible color band of the spectrum),while the IR pixel has a filter that cuts off light in the visiblespectrum and passes only light in the NIR or IR band or region of thespectrum. Optionally, for example, the IR filter may comprise a redfilter and a blue filter stacked one on top of the other to form a NIRor IR pass filter. The system may function to process an output of theNIR or IR pixel to determine an infrared component of the imaged sceneand may subtract the infrared component from the outputs of the red,green and blue pixels to obtain a more accurate or true color responsefor the pixels and to limit or avoid infrared color wash-out, asdiscussed below.

Optionally, the R-G-B-IR filter arrangement may be any suitablearrangement of pixels/filters in a repeating pattern, such as the 2×2arrangements of pixels shown in FIGS. 3A and 3B. As shown in FIG. 3A, a2×2 pixel matrix or sub-array may have the IR pixel 12 d positioneddiagonal to the red pixel 12 a. Since the red pixel has a stronger NIRresponse than the green pixel 12 c and the blue pixel 12 b, such anarrangement will generate a more evenly distributed IR image by placingthe IR pixel and the red pixel diagonally. Such an arrangement will alsobenefit the IR image resolution in the de-mosaicing methods describedbelow. Alternately, and as shown in FIG. 3B, the pixel arrangement mayhave a repeating 2×2 matrix or sub-array with the green and blue pixels12 c, 12 b along one column and the red and IR pixels 12 a, 12 d alongthe other column, such that half of the columns (green and blue pixelsonly) has a weaker IR sensitivity, while the other half of the columnshas a greater IR sensitivity. The benefit of such an arrangement is thedirect replacement (as compared to a typically R-G-G-B pixelarrangement) of a green pixel by an IR pixel (because a typicallyR-G-G-B pixel arrangement would have the IR pixel replaced with a greenpixel), which results in smaller imager fabrication process change. Inthe de-mosaicing technique discussions below, the de-mosaicingtechniques are described in relation to the pixel arrangement of FIG.3A, but it is envisioned that similar de-mosaicing techniques may becarried out with the pixel arrangement of FIG. 3B, while remainingwithin the spirit and scope of the present invention.

Since the red, green and blue pixels typically may have a similar IRresponse as the IR pixel, the establishment of the IR pixel on eachsub-array of the pixelated array allows for a determination of the IRcomponent present in the light being imaged. Thus, the IR component (theIR or NIR energy or light sensed by the IR pixel) can be subtracted fromthe red, green and blue pixel response signal to obtain a more accurateor true color response for the pixels and to limit or avoid IR colorwash-out. Further, with the IR cut-off in the sensor pixel level, thereis no need to add IR cut-off filter in a lens. This simplifies the lensdesign and results in significant cost savings in manufacturing of theoptical lens for the imager. Because the sensitivities in the IR regionof the red, green and blue pixels may be different, factors or ratiosmay be applied to the IR signal when subtracting the IR value orcomponent from the color signals. For example, the color values of thepixel signals may be determined via the following equations:

Red Color Value=Red Pixel Value−A _(R) *IR Pixel Value;

Green Color Value=Green Pixel Value−A _(G) *IR Pixel Value;

Blue Color Value=Blue Pixel Value−A _(B) *IR Pixel Value; and

IR Value=IR Pixel Value,

where A_(R), A_(G), and A_(B) are the selected factors for the redpixels, green pixels and blue pixels, respectively.

Optionally, for example, these factors can be selected to be 1 tosimplify the de-mosaicing circuitry or software. However, the spectralresponses of the R, G and B pixels in the NIR region may not match wellwith that of the IR pixel. As a result, it may be difficult to totallycancel out the IR signal from the R, G and B pixels that sensing lightin the visible range of the spectrum. Optionally, the factors of each R,G and B color can be determined by the corresponding spectralsensitivities of the sensor. The spectrum contribution of the lightsource should also be considered. For example, for applications wherethe scene is illuminated by a light source or light sources emittingillumination within a known spectral band or region of the spectrum, acombination of the light source and the imager's spectral sensitivitymay determine the factors. For example, if an imaged object orillumination source contains narrow-band IR illumination around 750 nm,the imager with spectral characteristics such as shown in FIG. 2 shouldhave different green and blue factor values. As another example, when anincandescent light source or a Halogen light source is used, because oftheir light output being in a more wide spread spectrum in both visibleand IR ranges, there should be another set of factors for the red, greenand blue pixels. Because of the known source spectrum in such cases, thefactors can be pre-determined. Also, if an application involves multiplecontrolled illumination spectra sources (with at least one of theillumination sources generating a light output having different spectralcharacteristics than at least one other of the illumination sources),the factors corresponding to each illumination source can be stored inthe memory of the imager or camera or processor and loaded to the imageror the processor for calibration when the illumination source changes toa different illumination source generating a light output having adifferent spectral characteristic.

Optionally, for illumination sources that generate a light or energyoutput with changing or un-controlled spectrum or spectralcharacteristics, such as sun light at different times of the day, moresophistical image processing can be applied to determine the factors.For example, an iterative approach can be used in which multiple passeswith different factors determine the best factors. As another example, acolor conversion matrix can be used in a manner that is similar to anauto white balance technique that is commonly used in visible lightcameras, wherein a statistical analysis of the scene image is performedto determine the contribution of individual visible colors and the IRspectrum and to determine the values of three factors.

With the pixel arrangements discussed above and the de-mosaicingtechniques described below, an imager or imaging system of the presentinvention can produce a visible color image and a NIR image at the sametime. The following sections describe the different de-mosaicingtechniques of the present invention.

De-Mosaicing Techniques

A pixelated imaging array of the present invention may have an R-G-B-IRpixel pattern, such as described above and shown in FIG. 3A. Althoughshown in FIGS. 4-7 as a 4×4 array of pixels, clearly the imaging arraymay have and preferably has an array of at least about 40×40 pixels andmay have an array of, for example, about 512×512 pixels or 640×480pixels or thereabouts or any other suitable array for capturing imagesin the field of view of the imaging sensor. In the illustratedembodiments, the 4×4 pixel sub-arrays are described below with theindividual pixels at respective coordinates corresponding to the row andcolumn location of the pixels (for example, the R1 red pixel is atlocation r0,c0, while the G3 green pixel is at location r2,c1, and soon). The de-mosaicing techniques of the present invention are applicableto any sized pixelated array and repeat the process over the entirearray to de-mosaicize the pixelated array. The following de-mosaicingformulas describe different methods of extracting red, green, blue andIR values for each pixel from itself and the adjacent pixels. Thus, avisible color image and a NIR image can be produced at the same time viathe processing techniques of the present invention.

2×2 Window De-Mosaicing

For an R-G-B-IR pixel pattern, each pixel of the array has itsassociated primary color (or is sensitive to a particular band of thespectrum and is less sensitive to other bands of the spectrum), whichcomes from its filter type. For example, and with reference to FIG. 5,the R1 red pixel at r0 c 0 has a primary color of red, so that the redvalue for this pixel is readily available, while the G1 green pixel atr0,c1 has a primary color of green, so that the green value for thispixel is readily available, and the B1 blue pixel at r1,c0 has a primarycolor of blue so that the blue value for this pixel is readilyavailable. The green, blue and IR values for this pixel location arecalculated from the adjacent pixels in a 2×2 window. Each window coversfour pixels in two rows and two columns and in order of red, green, blueand IR, such as shown in FIG. 5. The 2×2 window does not overlap anyadjacent 2×2 windows. As an example in the first 2×2 window, theformulas for calculating the color or IR values for the pixel locationsare as follows:

R(r0c0)=R1−A _(R) *IR1, G(r0c0)=G1−A _(G) *IR1, B(r0c0)=B1−A _(B) *IR1,and IR(r0c0)=IR1

R(r0c1)=R1−A _(R) *IR1, G(r0c1)=G1−A _(G) *IR1, B(r0c1)=B1−A _(B) *IR1,and IR(r0c1)=IR1

R(r1c0)=R1−A _(R) *IR1, G(r1c0)=G1−A _(G) *IR1, B(r1c0)=B1−A _(B) *IR1,and IR(r1c0)=IR1; and

R(r1c1)=R1−A _(R) *IR1, G(r1c1)=G1−A _(G) *IR1, B(r1c1)=B1−A _(B) *IR1,and IR(r1c1)=IR1,

where A_(R), A_(G), and A_(B) are the selected factors for the redpixels, green pixels and blue pixels, respectively, and where R(rx,cx)is the red color value for the respective pixel location, G(rx,cx) isthe green color value for the respective pixel location, B(rx,cx) is theblue color value for the respective pixel location, and IR(rx,cx) is theinfrared value for the respective pixel location, and where R1 is anoutput value of the red pixel of the window or sub-array, G1 is anoutput value of the green pixel of the window or sub-array, B1 is anoutput value of the blue pixel of the window or sub-array, and IR1 is anoutput value of the IR pixel of the window or sub-array.

Thus, the value of the IR component for that pixel location (asdetermined by multiplying the value of the IR pixel by a factoring valuefor each color) is subtracted from the value of the respective colorpixel (and the color for the adjacent pixels also has the factored IRvalue subtracted therefrom) to determine the color value of each pixellocation. This is repeated for each 2×2 window of the array.

2×2 Rolling Window De-Mosaicing

Unlike the 2×2 Window De-mosaicing method above, the 2×2 Rolling WindowDe-mosaicing process shifts the 2×2 window by one pixel at a time inboth the row and column directions, such as can be seen with referenceto FIG. 6. The following formulas show an example of de-mosaicing fourpixels of each color band or IR band:

R(r0c0)=R1−A _(R) *IR1, G(r0c0)=G1−A _(G) *IR1, B(r0c0)=B1−A _(B) *IR1,and IR(r0c0)=IR1

R(r0c1)=R2−A _(R) *IR1, G(r0c1)=G1−A _(G) *IR1, B(r0c1)=B2−A _(B) *IR1,and IR(r0c1)=IR1

R(r1c0)=R3−A _(R) *IR1, G(r1c0)=G3−A _(G) *IR1, B(r1c0)=B1−A _(B) *IR1,and IR(r1c0)=IR1; and

R(r1c1)=R4−A _(R) *IR1, G(r1c1)=G3−A _(G) *IR1, B(r1c1)=B2−A _(B) *IR1,and IR(r1c1)=IR1,

where A_(R), A_(G), and A_(B) are the selected factors for the redpixels, green pixels and blue pixels, respectively, and where R(rx,cx)is the red color value for the respective pixel location, G(rx,cx) isthe green color value for the respective pixel location, B(rx,cx) is theblue color value for the respective pixel location, and IR(rx,cx) is theinfrared value for the respective pixel location, and where Rx is anoutput value of the red pixel of a respective sub-array, Gx is an outputvalue of the green pixel of a respective sub-array, Bx is an outputvalue of the blue pixel of a respective sub-array and IRx is an outputvalue of the IR pixel of a respective sub-array.

Thus, the color values for the individual pixel locations are based onthe color values for the respective color pixel within the particular2×2 window, as reduced by the factored IR value of the IR pixel withinthe particular 2×2 window. The 2×2 window moves or shifts along the rowsand columns during the processing of the pixelated array.

3×3 Rolling Window De-Mosaicing

Optionally, the imaging array may be de-mosaiced via a 3×3 rollingwindow de-mosaicing technique. In this method, each pixel will have itsred, green, blue and IR information from averaging the adjacent pixelsexcept for the primary color itself, such as can be seen with referenceto FIG. 7. For example, the formulas for the pixel values at the pixellocations r1 c 1, r1 c 2, r2 c 1 and r2 c 2 are:

R(r1c1)=(R1+R2+R3+R4)/4−A _(R) *IR1, G(r1c1)=(G1+G3)/2−A _(G) *IR1,B(r1c1)=(B1+132)/2−A _(B) *IR1, and IR(r0c0)=IR1;

R(r1c2)=(R2+R4)/2−A _(R)*(IR1+1R2)/2, G(r1c2)=(G1+G2+G3+G4)/4−A_(G)*(IR1+IR2)/2, B(r0c1)=B2−A _(B)*(IR1+1R2)/2, andIR(r0c1)=(IR1+1R2)/2;

R(r2c1)=(R3+R4)/2−A _(R)*(IR1+1R3)/2, G(r2c1)=G3−A _(G)*(IR1+1R3)/2,B(r2c1)=(B1+132+63+64)/4−A _(B)*(IR1+1R3)/2, and IR(r2c1)=(IR1+1R3)/2;and

R(r2c2)=R4−A _(R)*(IR1+1R2+1R3+1R4)/4, G(r1c1)=(G3+G4)/2−A_(G)*(IR1+1R2+1R3+1R4)/4, B(r2c2)=(B2+134)/2−A _(B)*(IR1+1R2+1R3+1R4)/4,and IR(r2c2)=(IR1+1R2+1R3+1R4)/4,

where A_(R), A_(G), and A_(B) are the selected factors for the redpixels, green pixels and blue pixels, respectively, and where R(rx,cx)is the red color value for the respective pixel location, G(rx,cx) isthe green color value for the respective pixel location, B(rx,cx) is theblue color value for the respective pixel location, and IR(rx,cx) is theinfrared value for the respective pixel location, and where Rx is anoutput value of the red pixel of a respective sub-array, Gx is an outputvalue of the green pixel of a respective sub-array, Bx is an outputvalue of the blue pixel of a respective sub-array and IRx is an outputvalue of the IR pixel of a respective sub-array.

Thus, the values for each color component of each row/column location isdetermined by averaging the respective colored pixels that are adjacentto the particular pixel location or coordinate. For example, the redcolor component for the pixel location r1,c1 is the average of the fourred pixels R1, R2, R3, R4 (at r0,c0; r0,c2; r2,c0; and r2,c2) that areadjacent to the pixel location r1,c1, while the green color componentfor the pixel location r1,c1 is the average of the two green pixels G1,G3 (at r0,c1 and r2,c1) that are adjacent to the pixel location r1,c1,and the blue color component for the pixel location r1,c1 is the averageof the two blue pixels B1, B2 (at r1,c0 and r1,c2) that are adjacent tothe pixel location r1,c1. For the infrared value at the pixel locationr1,c1, the infrared value is determined by the value of the IR1 pixel atthat particular location. The color or IR values for the othercoordinates or pixel locations are determined in a similar manner.

Implementation Methods

It is envisioned that any of the above de-mosaicing techniques may beimplemented in conjunction with any suitable imaging array sensor. Forexample, the above de-mosaicing techniques can be implemented on or inconjunction with an imager chip, which is often called a system-on-chip(SOC). Optionally, such processing techniques may be implemented outsidethe imager chip by means of a FPGA, DSP or microprocessor or the like,while remaining within the spirit and scope of the present invention.

Applications IHC Reflective Object Identification

In Intelligent Headlamp Control (IHC) applications, spectralinformation, such as R, G, B color information may be used to assistidentifying tail lights, head lamps and traffic signs, such as describedin U.S. Pat. Nos. 5,796,094; 6,097,023; 6,320,176; 6,313,454; 6,559,435;6,831,261; 6,396,397; 5,877,897; 6,498,620; 5,670,935; 6,946,978;7,123,168; 7,004,606; 7,005,974 and/or 5,550,677, and/or PCT ApplicationNo. PCT/US07/75702, filed Aug. 10, 2007 and published Feb. 28, 2008 asInternational Publication No. WO 2008/024639, and/or U.S. patentapplication Ser. No. 11/239,980, filed Sep. 30, 2005, now U.S. Pat. No.7,881,496; and/or Ser. No. 10/534,632, filed May 11, 2005, now U.S. Pat.No. 7,965,336; and/or PCT Application No. PCT/US03/36177, filed Nov. 14,2003 and published Jun. 3, 2004 as International Publication No. WO2004/047421, which are hereby incorporated herein by reference in theirentireties. It is usually challenging to identify and separatereflective objects, like traffic signs in night scene, from tail lights,head lamps and street lights. The techniques that have been often usedare identifying color (traffic signs are often yellow or white) andshapes (traffic signs are often square or rectangle). However, thesetechniques are not always effective and accurate.

The present invention provides a processing technique that uses IRinformation to increase the identification accuracy. In addition tothree visible colors, IR adds one more dimension to the color space. Alltail lights, head lamps and traffic signs have their IR signatures basedon how much they emit or reflect IR light. For instance, a traffic signreflects the light from the head lamps of an approaching vehicle, withthe head lamp illumination carrying a known IR signature. QuantitativeIR signature of objects helps to determine the types of objects, such asfor example, whether an object is a light or a road side traffic sign orother object. The IR signatures include:

-   -   1. Ratios of IR to R signals of object types, IR to G signals of        object types, and IR to B signals of object types.    -   2. An extension to the 2-D CIE color space to form a 3-D space.        The location of an object type (such as a stop sign or the like)        in the 3-D space is a signature that identifies the object type.    -   3. R, G, B, IR 4-D coordinate space. The location of an object        type in the 4-D coordinate space is a signature that identifies        the object type.

In the signatures of the types 2 and 3 above, the vision algorithm cancalibrate and define an area, range or segment of these color spaces toclassify the object types located inside these regions. Thus, the systemmay recognize or identify an object based on whether the signature ofthe detected object falls within a particular signature range that ispreviously categorized or identified as being associated with orindicative of or representative of a particular object or type or classof object.

The system thus may utilize the CIE color space, as is known in thecolor art, such as by utilizing aspects of the systems described in U.S.patent application Ser. No. 11/239,980, filed Sep. 30, 2005, now U.S.Pat. No. 7,881,496, which is hereby incorporated herein by reference inits entirety. The Centre Internationale de l'Eclairage (CIE) is aninternational organization that establishes methods for measuring colorand that recommends standards and procedures for light and lighting,including colorimetry. The color standards for colormetric measurementsare internationally accepted specifications that define color valuesmathematically. CIE defines color as a combination of three axes: x, y,and z. CIE color models are considered device independent because thecolors should not differ, theoretically, from one output device toanother if properly calibrated.

In addition to the spectral domain, one can apply IR processingtechniques to identify an object in the time domain. For example, avehicle can include an IR light source, such as an IR LED or the like,at or near or in the headlamp assembly. The headlamp's IR output can bemodulated with a pulse width modulation (PWM) sequence. Since onlytraffic signs, not the head lamps of other oncoming vehicles or taillights of other leading vehicles reflect the IR light, the traffic signsappear in the image frames that the headlamp IR source is turned on, andis diminished or reduced in the image frames that the headlamp IR sourceis turned off. For the objects seen in the IR image that possibly comefrom oncoming vehicle headlamps, proceeding vehicle tail lights orstreet lights, the captured images of these objects or illuminationsources will not have such an on and off sequence, which is synchronizedwith the IR light source PWM sequence. Such a processing technique canalso help identify and separate the reflective objects in the forwardfield of view of the imager, such as retro-reflective marks/signs thatare placed on the mailbox, front yard, building side or reflectivestripes on trucks. The modulated IR light is not visible or discernibleto humans, so it does not impede drivers of other vehicles or the driverof the subject or host vehicle and does not violate related regulations.The PWM sequence of each vehicle can be encoded or controlled withunique “signature” to avoid the interference from other vehiclesequipped with the same type of PWM IR lights. Optionally, for example,this signature can be one of the following or a combination of thefollowing: (1) unique modulation sequence, (2) unique modulationfrequency.

NIR Night Vision Capability in Forward Viewing Camera

A NIR night vision system of the present invention may include a NIRlight source (such as a LED, laser, or other NIR light sources) thatilluminates the scene in the dark environment, a NIR video camera thatcaptures image/video of the scene, and a display that shows the capturedNIR images to a driver. For example, and with reference to FIG. 8, avision system 110 includes an R-G-B-IR imager or imaging device orcamera 112, a NIR illuminator or illumination source 114 (such as a NIRLED or the like) and a display device 116. The imager may include or maybe associated with a lens or optic 118 for focusing the scene onto animaging plane of the imager. The system 110 includes an R-G-B-IRde-mosaicing processor or algorithm 120 or the like, which provides orgenerates an output, such as for displaying images at the display device116 or for further processing by an associated forward viewing visionsystem 122 (which may be part of or associated with the de-mosaicingprocessor or a separate processor or the like), such as a headlampcontrol system or a lane departure warning system or the like.

The NIR night vision system allows a driver to see a farther distanceahead of the vehicle and to see and discern more objects than he or shecan normally see with typical head lamp illumination. Such a visionsystem thus gives the driver more time to react to different drivingconditions. It is desirable to have a NIR camera mounted inside thevehicle compartment and looking forward. To avoid blocking the viewingangle of driver, it is best to mount the camera behind the rear viewmirror, and preferably with a forward field of view through thewindshield of the vehicle and through an area that is wiped or cleanedby the windshield wipers of the vehicle. The NIR illumination source canbe either mounted inside one or both headlamps or inside the drivercompartment, such as at or in or near or behind the interior rear viewmirror or accessory module or windshield electronics module or the like.

The forward viewing camera thus may be mounted behind the rear viewmirror and inside the vehicle, and the system may provide driverassistance and safety functions, such as intelligent headlamp control,lane departure warning, traffic sign recognition, adaptive cruisecontrol, and/or the like. Optionally, the forward viewing camera mayprovide or be associated with a rain sensing function, or there may beother sensing devices, such as a rain sensor or the like, mounted at ornear or behind the rear view mirror. Adding a NIR camera to an alreadycrowded space behind the rear view mirror can be challenging and costly.It is thus desirable to combine the NIR camera to a visible forwardviewing camera to save space and cost. The R-G-B-IR imager of thepresent invention provides a solution to combine the visible forwardviewing camera and the NIR camera into a single imager and camera body.

Rear Viewing Camera with NIR Night Vision

A typical automotive rear viewing camera includes a color imager and alens with IR cut-off filter. The IR cut-off filter in the lens preventsthe camera from IR contamination and provides good visible color imagesfor the camera in normal lighting conditions. However, in a nightcondition when no or minimum ambient light is available to illuminatethe scene behind the vehicle, the only illumination comes from thevehicle tail lights. The tail lights, which include running lights,brake lights and back up lights, typically produce illumination of lessthan a few Lux. Thus, in a rear viewing camera image with an IR cut-offfilter, the night scene image is mostly noisy, dark and low-contrast.One can thus only view a very short distance behind the vehicle. This ispartially because the IR cut-off filter cuts off a significant amount ofthe existing IR light in the tail light illumination which otherwiseassists the imaging sensitivity.

To provide enhanced safety and enhanced convenience to the drivers, itis desirable to increase the low light sensitivity of the rear viewingcamera. It may be desirable to take off the IR cutoff filter from thelens, so that the scene illuminated by the near IR light component fromthe tail lights, street light, garage light, and/or the like can bebetter imaged by the imager, since incandescent light bulbs, which aretypically used in vehicle tail lights, street lights, garage lights, andthe like, emit a relatively high amount of near IR light. It may also bedesired to include an additional invisible near IR light source (suchas, for example, a near infrared LED or the like) at or near or in thetail light assembly for illuminating the rearward scene of the vehicle.The high sensitivity of silicon imager in the near IR region helpsincrease the image sensitivity of the night scene.

The R-G-B-IR imager described above provides a possibility of using NIRto enhance low light sensitivity of rear viewing cameras. A rear viewingcamera, such as shown in FIG. 9, can provide good color images in normallighting condition and enhanced low light sensitivity in low light ornighttime conditions. In a low light night scene, it is of lessimportance to produce true color images (because human eyes do notdistinguish color as well under low illumination conditions) so ablack-and-white image is sufficient for the night scene display. Ahardware switch can be used to switch the video signal that feeds thedisplay between the normal lighting color image and low light NIR image,such as by utilizing aspects of the imaging systems described in U.S.Pat. No. 6,498,620; and/or U.S. patent application Ser. No. 11/239,980,filed Sep. 30, 2005, now U.S. Pat. No. 7,881,496; and/or Ser. No.10/534,632, filed May 11, 2005, now U.S. Pat. No. 7,965,336; and/or PCTApplication No. PCT/US03/36177, filed Nov. 14, 2003 and published Jun.3, 2004 as International Publication No. WO 2004/047421, which arehereby incorporated herein by reference in their entireties.

For example, and with reference to FIG. 9, a NIR night vision system110′ of the present invention may include an R-G-B-IR imager or imagingdevice or camera 112 and associated lens or optic 118, a NIR illuminatoror illumination source 114 (such as a NIR LED or the like) and a displaydevice 116. The system 110′ includes an R-G-B-IR de-mosaicing processoror algorithm 120′ or the like, which provides or generates an output(such as an RGB color output and an IR output), and the system mayinclude a normal and low light image switch 124 that determines whetheror not to display the captured color images or the captured IR or NIRimages at the display device 116. The switch may adjust the displayedimages in response to the ambient light level at the scene being aboveor below a threshold level.

To further increase the signal strength and sensitivity of the low lightimages, it is desirable to use all the available photons that the imagercan detect in the NIR and visible wavelength ranges. The red and IRpixels are more sensitive to the light sources in low light nightconditions, which includes incandescent bulb tail lights, an added NIRilluminator, garage lamp lights, street lights, and the like, than greenand blue pixels, since most of these lights have stronger red and NIRspectrum range components than green and blue spectrum range components.To take advantage of this feature, a de-mosaicing method of the presentinvention may be implemented to extract the desired image colorcomponents. As can be seen with reference to FIG. 10, the pixel valuesfrom the green and blue pixels are not used for the image processing inthis method. Instead, the combined values coming from the surroundingred and IR pixels are used to represent the pixel values of green andblue pixels. In this method, the imager is treated as a substantiallymonochrome imager. Each pixel produces only one grey scale value.

The switching between normal color image and low light monochrome imagemay be done in an imager de-mosaicing unit. For example, and withreference to FIG. 11, a rear viewing camera system 110″ may include anR-G-B-IR imager or imaging device or camera 112 and associated lens oroptic 118, a NIR illuminator or illumination source 114 (such as a NIRLED or the like) and a display device 116. The system 110″ includes anR-G-B-IR de-mosaicing processor or algorithm 120″, which includes animage mode switch such as described above, and which provides orgenerates an output (such as an RGB color output or color image signaland a monochrome output or monochrome image signal), whereby the displaydevice 116 displays the color image or monochrome image in response tothe de-mosaicing unit.

Optionally, the imaging sensor may include a plurality of sub-arrays,such as a plurality of 2×2 sub-arrays or the like, that may eachcomprise a principally red sensing pixel, a principally blue sensingpixel, a principally green sensing pixel and a principally IR or near IRsensing pixel, as discussed above, or may include other types ofsub-arrays having different arrangements of pixels such as one or moreprincipally IR or NIR sensing pixels and one or more color sensingpixels. For example, an imaging sensor may include a plurality ofsub-arrays where each sub-array may comprise a color sensing pixel (suchas a pixel primarily sensing red light from the red region of thespectrum, blue light from the blue region of the spectrum or green lightfrom the green region of the spectrum, such as may be achieved byplacing a color filter in front of or over the respective pixel) and aclear pixel (that has a clear spectral filter or no spectral filter andis receptive to all or substantially all incoming visible and infraredlight or energy incident thereon) and a principally IR or NIR sensingpixel (such as a principally IR or NIR sensing pixel as describedabove). Such an imaging sensor may function in a similar manner asdescribed above, where the IR value (as sensed by the principally IR orNIR sensing pixel) may be subtracted by the accompanying image processorfrom the color value sensed by the color pixel and the clear valuesensed by the clear pixel to obtain a more accurate or true colorresponse for the color and clear pixels and to limit or avoid infraredcolor wash-out. Clearly, other arrangements of pixels and filters, suchas an imaging sensor having a sub-array having one or more color sensingpixels and/or one or more clear pixels and one or more principally IR orNIR sensing pixels, such as an imaging sensor having three pixelsub-arrays or 2×2 sub-arrays or 3×3 sub-arrays or the like, may beimplemented with one or more IR or NIR sensing pixels while remainingwithin the spirit and scope of the present invention.

A benefit of the present invention is that a principally IR or NIRsensing pixel is in close proximity and optionally contiguous with theassociated cluster of spectrally selective and/or clear pixels (such asred, green and/or blue pixels and/or clear pixels), so that the imagedata associated with a particular object in the field of view beingimaged (such as, for example, a headlamp or taillight in a forward fieldof view or a child or object in a rearward field of view or a vehicle orobject in a forward field of view or a child or seat occupant in aninterior cabin field of view or the like) can be processed to subtractout or account for the IR or NIR content of light or energy in the fieldof view by subtracting out the IR or NIR values (as sensed by theprincipally IR or NIR sensing pixel) from the other pixels of aparticular sub-array of pixels (that images the headlamp or taillight orobject or child or occupant that is in the field of view of the imagingsensor) to enhance the image output and/or to obtain a more accurate ortrue color response for the pixels and to limit or avoid infrared colorwash-out. Thus, the IR or NIR values local to or associated with anillumination source or object being sensed by a respective sub-array ofthe imaging sensor may be subtracted from the color values or clearspectral values sensed by the color and/or clear pixels of thatrespective sub-array (with the color and/or clear pixels of thesub-array being proximate to or associated with the principally IR orNIR sensing pixel or pixels of that sub-array), so that the system mayprovide enhanced detection of and/or identification of the illuminationsources or objects exterior or interior of the vehicle that are beingimaged by the imaging sensor and system.

Optionally, the imaging sensor may be suitable for various forwardimaging systems, such as for headlamp control systems, rain sensorsystems, lane departure warning systems, object detection systems,and/or the like. For example, the imaging sensor of the presentinvention may be associated with one or more systems of the typesdescribed in U.S. Pat. Nos. 5,796,094; 6,396,397; 6,806,452; 6,690,268;7,005,974; 7,123,168; 7,004,606; 6,946,978; 7,038,577; 6,353,392;6,320,176; 6,313,454 and 6,824,281, which are all hereby incorporatedherein by reference in their entireties.

Optionally, the imaging sensor may be suitable for various in-cabinetimaging systems, such as driver drowsiness monitoring, passengeridentification for intelligent airbag deployment, back seat passengermonitoring, and/or the like.

Optionally, the imaging sensor may be suitable for use in connectionwith other vehicle imaging systems, such as, for example, a blind spotdetection system, where a blind spot indicator may be operable toprovide an indication to the driver of the host vehicle that an objector other vehicle has been detected in the lane or area adjacent to theside of the host vehicle. In such a blind spot detector/indicatorsystem, the blind spot detection system may include an imaging sensor orsensors, or ultrasonic sensor or sensors, or sonar sensor or sensors orthe like. For example, the blind spot detection system may utilizeaspects of the blind spot detection and/or imaging and/or indicatingsystems described in U.S. Pat. Nos. 7,038,577; 6,882,287; 6,198,409;5,929,786 and/or 5,786,772, and/or U.S. patent application Ser. No.11/315,675, filed Dec. 22, 2005, now U.S. Pat. No. 7,720,580; Ser. No.11/239,980, filed Sep. 30, 2005, now U.S. Pat. No. 7,881,496; and/orSer. No. 11/933,697, filed Nov. 1, 2007, now U.S. Pat. No. 7,777,611,and/or PCT Application No. PCT/US06/26148, filed Jul. 5, 2006 andpublished Jan. 11, 2007 as International Publication No. WO 2007/005942,and/or PCT Application No. PCT/US07/082099, filed Oct. 22, 2007 andpublished May 2, 2008 as International Publication No. WO 2008/051910,and/or U.S. provisional applications, Ser. No. 60/618,686, filed Oct.14, 2004; Ser. No. 60/853,850, filed Oct. 24, 2006; Ser. No. 60/918,089,filed Mar. 15, 2007; Ser. No. 60/970,687, filed Sep. 7, 2007; and/orSer. No. 60/857,025, filed Nov. 6, 2006, and/or of the reverse or backupaid systems, such as the rearwardly directed vehicle vision systemsdescribed in U.S. Pat. Nos. 5,550,677; 5,760,962; 5,670,935; 6,201,642;6,396,397; 6,498,620; 6,717,610; 6,757,109 and/or 7,005,974, and/or ofthe rain sensors described in U.S. Pat. Nos. 6,250,148 and 6,341,523,and/or of other imaging systems, such as the types described in U.S.Pat. Nos. 7,123,168; 6,353,392 and 6,313,454, with all of the abovereferenced U.S. patents, patent applications and provisionalapplications and PCT applications being commonly assigned and beinghereby incorporated herein by reference in their entireties.

Optionally, the mirror assembly and/or accessory module or windshieldelectronics module may include one or more displays, such as fordisplaying the captured images or video images captured by the imagingsensor or sensors of the vehicle, such as the displays of the typesdisclosed in U.S. Pat. Nos. 7,004,593; 5,530,240 and/or 6,329,925, whichare hereby incorporated herein by reference, and/or display-on-demand ortransflective type displays, such as the types disclosed in U.S. Pat.Nos. 7,274,501; 7,255,451; 7,195,381; 7,184,190; 6,690,268; 5,668,663and/or 5,724,187, and/or in U.S. patent application Ser. No. 10/538,724,filed Jun. 13, 2005 and published Mar. 9, 2006 as U.S. Publication No.US 2006-0050018; Ser. No. 11/226,628, filed Sep. 14, 2005 and publishedMar. 23, 2006 as U.S. Publication No. US 2006-0061008; Ser. No.10/993,302, filed Nov. 19, 2004, now U.S. Pat. No. 7,338,177; and/orSer. No. 11/284,543, filed Nov. 22, 2005, now U.S. Pat. No. 7,370,983,and/or PCT Application No. PCT/US03/29776, filed Sep. 9, 2003 andpublished Apr. 1, 2004 as International Publication No. WO 2004/026633;and/or PCT Application No. PCT/US03/35381, filed Nov. 5, 2003 andpublished May 21, 2004 as International Publication No. WO 2004/042457,and/or PCT Application No. PCT/US03/40611, filed Dec. 19, 2003 andpublished Jul. 15, 2004 as International Publication No. WO 2004/058540,and/or PCT Application No. PCT/US06/18567, filed May 15, 2006 andpublished Nov. 23, 2006 as International Publication No. WO 2006/124682,which are all hereby incorporated herein by reference in theirentireties, or may include or incorporate video displays or the like,such as the types described in PCT Application No. PCT/US03/40611, filedDec. 19, 2003 and published Jul. 15, 2004 as International PublicationNo. WO 2004/058540, and/or PCT Application No. PCT/US07/042718, filedOct. 31, 2006 and published May 10, 2007 as International PublicationNo. WO 07/053710, and/or U.S. patent application Ser. No. 10/538,724,filed Jun. 13, 2005 and published Mar. 9, 2006 as U.S. Publication No.US 2006-0050018; and/or Ser. No. 11/284,543, filed Nov. 22, 2005, nowU.S. Pat. No. 7,370,983, and/or U.S. provisional applications, Ser. No.60/732,245, filed Nov. 1, 2005; Ser. No. 60/759,992, filed Jan. 18,2006; and/or Ser. No. 60/836,219, filed Aug. 8, 2006, which are herebyincorporated herein by reference.

The imaging sensor may be incorporated at or in an accessory module orwindshield electronics module (such as described above), or may beincorporated at or in an interior rearview mirror assembly of thevehicle, while remaining within the spirit and scope of the presentinvention. Optionally, the accessory module may utilize aspects of otheraccessory modules or windshield electronics modules or the like, such asthe types described in U.S. patent application Ser. No. 11/201,661,filed Aug. 11, 2005, and/or U.S. Pat. Nos. 7,188,963; 7,004,593;6,824,281; 6,690,268; 6,250,148; 6,341,523; 6,593,565; 6,428,172;6,501,387; 6,329,925 and 6,326,613, and/or in PCT Application No.PCT/US03/40611, filed Dec. 19, 2003 and published Jul. 15, 2004 asInternational Publication No. WO 2004/058540, and/or Ireland pat.applications, Ser. No. S2004/0614, filed Sep. 15, 2004; Ser. No.S2004/0838, filed Dec. 14, 2004; and Ser. No. S2004/0840, filed Dec. 15,2004, which are all hereby incorporated herein by reference.

Optionally, the mirror assembly and/or module may support one or moreother accessories or features, such as one or more electrical orelectronic devices or accessories. For example, illumination sources orlights, such as map reading lights or one or more other lights orillumination sources, such as illumination sources of the typesdisclosed in U.S. Pat. Nos. 6,690,268; 5,938,321; 5,813,745; 5,820,245;5,673,994; 5,649,756; 5,178,448; 5,671,996; 4,646,210; 4,733,336;4,807,096; 6,042,253; 6,971,775; 7,249,860; 7,195,381 and/or 5,669,698,which are hereby incorporated herein by reference in their entireties,may be included in the mirror assembly or module. The illuminationsources and/or the circuit board may be connected to one or more buttonsor inputs for activating and deactivating the illumination sources.Optionally, the mirror assembly or module may also or otherwise includeother accessories, such as microphones, such as analog microphones ordigital microphones or the like, such as microphones of the typesdisclosed in U.S. Pat. Nos. 6,243,003; 6,278,377 and/or 6,420,975,and/or in U.S. patent application Ser. No. 10/529,715, filed Mar. 30,2005. Optionally, the mirror assembly may also or otherwise includeother accessories, such as a telematics system, speakers, antennas,including global positioning system (GPS) or cellular phone antennas,such as disclosed in U.S. Pat. No. 5,971,552, a communication module,such as disclosed in U.S. Pat. No. 5,798,688, a voice recorder,transmitters and/or receivers, such as for a garage door opener or avehicle door unlocking system or the like (such as a remote keylessentry system), a digital network, such as described in U.S. Pat. No.5,798,575, a memory mirror system, such as disclosed in U.S. Pat. No.5,796,176, a hands-free phone attachment, a video device for internalcabin surveillance (such as for sleep detection or driver drowsinessdetection or the like) and/or video telephone function, such asdisclosed in U.S. Pat. Nos. 5,760,962 and/or 5,877,897, a remote keylessentry receiver, a seat occupancy detector, a remote starter control, ayaw sensor, a clock, a carbon monoxide detector, status displays, suchas displays that display a status of a door of the vehicle, atransmission selection (4wd/2wd or traction control (TCS) or the like),an antilock braking system, a road condition (that may warn the driverof icy road conditions) and/or the like, a trip computer, a tirepressure monitoring system (TPMS) receiver (such as described in U.S.Pat. Nos. 6,124,647; 6,294,989; 6,445,287; 6,472,979 and/or 6,731,205;and/or U.S. patent application Ser. No. 11/232,324, filed Sep. 21, 2005,now U.S. Pat. No. 7,423,522, and/or an ONSTAR® system and/or any otheraccessory or circuitry or the like (with all of the above-referencedpatents and PCT and U.S. patent applications being commonly assigned toDonnelly Corporation, and with the disclosures of the referenced patentsand patent applications being hereby incorporated herein by reference intheir entireties).

Changes and modifications to the specifically described embodiments maybe carried out without departing from the principles of the presentinvention, which is intended to be limited by the scope of the appendedclaims, as interpreted in accordance with the principles of patent law.

1. An imaging system suitable for use in a vehicle, said imaging systemcomprising: a camera comprising an imaging sensor; wherein said imagingsensor comprises a two-dimensional array of photosensing pixels; whereina red-pass spectral filter is disposed at each photosensing pixel of afirst set of photosensing pixels of said two-dimensional array, wherebysaid first set of photosensing pixels primarily senses red visiblelight; wherein a blue-pass spectral filter is disposed at eachphotosensing pixel of a second set of photosensing pixels of saidtwo-dimensional array, whereby said second set of photosensing pixelsprimarily senses blue visible light; wherein a green-pass spectralfilter is disposed at each photosensing pixel of a third set ofphotosensing pixels of said two-dimensional array, whereby said thirdset of photosensing pixels primarily senses green visible light; whereineach photosensing pixel of a fourth set of photosensing pixels of saidtwo-dimensional array senses at least near infrared radiation; whereinsaid camera is configured for attachment at an in-cabin side of awindshield of a vehicle equipped with said imaging system, and wherein,with said camera attached at the equipped vehicle, said camera has afield of view forward of the equipped vehicle through the windshield ofthe equipped vehicle; an image processor operable for processing imagedata captured by said imaging sensor; wherein said image data comprisesoutput of said first set of photosensing pixels of said two-dimensionalarray, output of said second set of photosensing pixels of saidtwo-dimensional array, output of said third set of photosensing pixelsof said two-dimensional array and output of said fourth set ofphotosensing pixels of said two-dimensional array; wherein said imagingsystem is associated with (i) a lane departure warning system of theequipped vehicle, (ii) an object detection system of the equippedvehicle and (iii) a night vision system of the equipped vehicle; a nearinfrared illumination source disposed at a front portion of the equippedvehicle and operable to emit non-visible near infrared illumination toilluminate non-visibly forward of the equipped vehicle to enhancenighttime viewing by said camera, and wherein said near infraredillumination source, when operated, illuminates at least a portion ofthe field of view of said camera with non-visible near infraredillumination; wherein outputs of said photosensing pixels of said first,second and third sets of photosensing pixels of said two-dimensionalarray of photosensing pixels are processed for at least one of (i) thelane departure warning system of the equipped vehicle and (ii) theobject detection system of the equipped vehicle; and wherein output ofsaid photosensing pixels of said fourth set of photosensing pixels ofsaid two-dimensional array of photosensing pixels is processed for thenight vision system of the equipped vehicle.
 2. The imaging system ofclaim 1, wherein said near infrared illumination source is disposed in aheadlamp assembly of the equipped vehicle.
 3. The imaging system ofclaim 2, wherein said near infrared illumination source, when operated,emits non-visible near infrared illumination that is modulated by pulsewidth modulation (PWM).
 4. The imaging system of claim 3, wherein imagedata capture by said imaging sensor is synchronized with modulation ofthe non-visible near infrared illumination emitted by said near infraredillumination source of the headlamp assembly.
 5. The imaging system ofclaim 3, wherein the non-visible near infrared illumination emitted bysaid near infrared illumination source of the headlamp assembly isencoded with a modulation sequence unique to the equipped vehicle toavoid other vehicles traveling ahead of the equipped vehicle interferingwith said imaging system.
 6. The imaging system of claim 1, wherein saidimaging system is operable to generate color video images based oncaptured image data.
 7. The imaging system of claim 1, wherein said nearinfrared illumination source comprises a near infrared light emittingdiode.
 8. The imaging system of claim 1, wherein a red filter and a bluefilter are stacked one on top of the other to form a filter that blocksvisible light at each photosensing pixel of said fourth set ofphotosensing pixels of said two-dimensional array.
 9. The imaging systemof claim 1, wherein said imaging system is associated with anintelligent headlamp control system of the equipped vehicle.
 10. Theimaging system of claim 1, wherein said near infrared illuminationsource comprises a near infrared emitting laser.
 11. The imaging systemof claim 1, wherein, responsive at least in part to processing by saidimage processor of output of said first set of photosensing pixels andwith said camera attached at the equipped vehicle and viewing forwardthrough the windshield of the equipped vehicle, said imaging system isoperable to detect traffic signs.
 12. The imaging system of claim 1,wherein, responsive at least in part to processing by said imageprocessor of image data captured by said imaging sensor and with saidcamera attached at the equipped vehicle and viewing forward of theequipped vehicle, said imaging system is operable to detect objectslocated external of the equipped vehicle.
 13. An imaging system suitablefor use in a vehicle, said imaging system comprising: a cameracomprising an imaging sensor; wherein said imaging sensor comprises atwo-dimensional array of photosensing pixels; wherein a red-passspectral filter is disposed at each photosensing pixel of a first set ofphotosensing pixels of said two-dimensional array, whereby said firstset of photosensing pixels primarily senses red visible light; wherein ablue-pass spectral filter is disposed at each photosensing pixel of asecond set of photosensing pixels of said two-dimensional array, wherebysaid second set of photosensing pixels primarily senses blue visiblelight; wherein a green-pass spectral filter is disposed at eachphotosensing pixel of a third set of photosensing pixels of saidtwo-dimensional array, whereby said third set of photosensing pixelsprimarily senses green visible light; wherein each photosensing pixel ofa fourth set of photosensing pixels of said two-dimensional array sensesat least near infrared radiation; wherein said camera is configured forattachment at an in-cabin surface of a windshield of a vehicle equippedwith said imaging system, and wherein, with said camera attached at theequipped vehicle, said camera has a field of view forward of theequipped vehicle through the windshield of the equipped vehicle; animage processor operable for processing image data captured by saidimaging sensor; wherein said image data comprises output of said firstset of photosensing pixels of said two-dimensional array, output of saidsecond set of photosensing pixels of said two-dimensional array, outputof said third set of photosensing pixels of said two-dimensional arrayand output of said fourth set of photosensing pixels of saidtwo-dimensional array; wherein said imaging system is associated with atleast one of (i) a lane departure warning system of the equippedvehicle, (ii) an object detection system of the equipped vehicle and(iii) an intelligent headlamp control system of the equipped vehicle;wherein said imaging system is associated with a night vision system ofthe equipped vehicle; a near infrared illumination source disposed at aportion of the equipped vehicle and operable to emit non-visible nearinfrared illumination to illuminate non-visibly forward of the equippedvehicle to enhance nighttime viewing by said camera, and wherein saidnear infrared illumination source, when operated, illuminates at least aportion of the field of view of said camera with non-visible nearinfrared illumination; wherein outputs of said photosensing pixels ofsaid first, second and third sets of photosensing pixels of saidtwo-dimensional array of photosensing pixels are processed for the atleast one of (i) the lane departure warning system of the equippedvehicle, (ii) the object detection system of the equipped vehicle and(iii) the intelligent headlamp control system of the equipped vehicle;wherein output of said photosensing pixels of said fourth set ofphotosensing pixels of said two-dimensional array of photosensing pixelsis processed for the night vision system of the equipped vehicle; andwherein said near infrared illumination source, when operated, emitsnon-visible near infrared illumination that is encoded by modulation.14. The imaging system of claim 13, wherein the non-visible nearinfrared illumination emitted by said near infrared illumination source,when said near infrared illumination source is operated, is modulated bypulse width modulation (PWM).
 15. The imaging system of claim 14,wherein image data capture by said imaging sensor is synchronized withmodulation of the non-visible near infrared illumination emitted by saidnear infrared illumination source.
 16. The imaging system of claim 13,wherein the non-visible near infrared illumination emitted by said nearinfrared illumination source is encoded with a modulation sequenceunique to the equipped vehicle to avoid other vehicles traveling aheadof the equipped vehicle interfering with said imaging system.
 17. Theimaging system of claim 16, wherein said imaging system is operable togenerate color video images based on captured image data.
 18. Theimaging system of claim 17, wherein said near infrared illuminationsource comprises a near infrared light emitting diode.
 19. The imagingsystem of claim 17, wherein said near infrared illumination sourcecomprises a near infrared emitting laser.
 20. The imaging system ofclaim 13, wherein said near infrared illumination source is disposed ata front portion of the equipped vehicle.
 21. The imaging system of claim13, wherein said near infrared illumination source is disposed in aheadlamp assembly of the equipped vehicle.
 22. The imaging system ofclaim 13, wherein said near infrared illumination source is disposed atthe in-cabin side of the windshield of the equipped vehicle.
 23. Animaging system suitable for use in a vehicle, said imaging systemcomprising: a camera comprising an imaging sensor; wherein said imagingsensor comprises a two-dimensional array of photosensing pixels; whereina red-pass spectral filter is disposed at each photosensing pixel of afirst set of photosensing pixels of said two-dimensional array, wherebysaid first set of photosensing pixels primarily senses red visiblelight; wherein a blue-pass spectral filter is disposed at eachphotosensing pixel of a second set of photosensing pixels of saidtwo-dimensional array, whereby said second set of photosensing pixelsprimarily senses blue visible light; wherein a green-pass spectralfilter is disposed at each photosensing pixel of a third set ofphotosensing pixels of said two-dimensional array, whereby said thirdset of photosensing pixels primarily senses green visible light; whereineach photosensing pixel of a fourth set of photosensing pixels of saidtwo-dimensional array senses at least near infrared radiation; whereinsaid camera is configured for attachment at an in-cabin surface of awindshield of a vehicle equipped with said imaging system, and wherein,with said camera attached at the equipped vehicle, said camera has afield of view forward of the equipped vehicle through the windshield ofthe equipped vehicle; an image processor operable for processing imagedata captured by said imaging sensor; wherein said image data comprisesoutput of said first set of photosensing pixels of said two-dimensionalarray, output of said second set of photosensing pixels of saidtwo-dimensional array, output of said third set of photosensing pixelsof said two-dimensional array and output of said fourth set ofphotosensing pixels of said two-dimensional array; wherein said imagingsystem is associated with at least one of (i) a lane departure warningsystem of the equipped vehicle, (ii) an object detection system of theequipped vehicle and (iii) an intelligent headlamp control system of theequipped vehicle; wherein said imaging system is associated with a nightvision system of the equipped vehicle; a near infrared illuminationsource disposed at a portion of the equipped vehicle and operable toemit non-visible near infrared illumination to illuminate non-visiblyforward of the equipped vehicle to enhance nighttime viewing by saidcamera, and wherein said near infrared illumination source, whenoperated, illuminates at least a portion of the field of view of saidcamera with non-visible near infrared illumination; wherein outputs ofsaid photosensing pixels of said first, second and third sets ofphotosensing pixels of said two-dimensional array of photosensing pixelsare processed for the at least one of (i) the lane departure warningsystem of the equipped vehicle, (ii) the object detection system of theequipped vehicle and (iii) the intelligent headlamp control system ofthe equipped vehicle; wherein output of said photosensing pixels of saidfourth set of photosensing pixels of said two-dimensional array ofphotosensing pixels is processed for the night vision system of theequipped vehicle; wherein said near infrared illumination source, whenoperated, emits non-visible near infrared illumination that ismodulated; and wherein the non-visible near infrared illuminationemitted by said near infrared illumination source is encoded with amodulation sequence unique to the equipped vehicle to avoid othervehicles traveling ahead of the equipped vehicle interfering with saidimaging system.
 24. The imaging system of claim 23, wherein thenon-visible near infrared illumination emitted by said near infraredillumination source, when said near infrared illumination source isoperated, is modulated by pulse width modulation (PWM).
 25. The imagingsystem of claim 24, wherein image data capture by said imaging sensor issynchronized with modulation of the non-visible near infraredillumination emitted by said near infrared illumination source.
 26. Theimaging system of claim 23, wherein said near infrared illuminationsource comprises a near infrared light emitting diode.
 27. The imagingsystem of claim 23, wherein said near infrared illumination sourcecomprises a near infrared emitting laser.
 28. The imaging system ofclaim 23, wherein said near infrared illumination source is disposed ata front portion of the equipped vehicle.
 29. The imaging system of claim28, wherein said near infrared illumination source is disposed in aheadlamp assembly of the equipped vehicle.
 30. The imaging system ofclaim 23, wherein said near infrared illumination source is disposed atthe in-cabin side of the windshield of the equipped vehicle.